5 research outputs found
Incremental Learning of Humanoid Robot Behavior from Natural Interaction and Large Language Models
Natural-language dialog is key for intuitive human-robot interaction. It can
be used not only to express humans' intents, but also to communicate
instructions for improvement if a robot does not understand a command
correctly. Of great importance is to endow robots with the ability to learn
from such interaction experience in an incremental way to allow them to improve
their behaviors or avoid mistakes in the future. In this paper, we propose a
system to achieve incremental learning of complex behavior from natural
interaction, and demonstrate its implementation on a humanoid robot. Building
on recent advances, we present a system that deploys Large Language Models
(LLMs) for high-level orchestration of the robot's behavior, based on the idea
of enabling the LLM to generate Python statements in an interactive console to
invoke both robot perception and action. The interaction loop is closed by
feeding back human instructions, environment observations, and execution
results to the LLM, thus informing the generation of the next statement.
Specifically, we introduce incremental prompt learning, which enables the
system to interactively learn from its mistakes. For that purpose, the LLM can
call another LLM responsible for code-level improvements of the current
interaction based on human feedback. The improved interaction is then saved in
the robot's memory, and thus retrieved on similar requests. We integrate the
system in the robot cognitive architecture of the humanoid robot ARMAR-6 and
evaluate our methods both quantitatively (in simulation) and qualitatively (in
simulation and real-world) by demonstrating generalized incrementally-learned
knowledge.Comment: This work has been submitted to the IEEE for possible publication.
Copyright may be transferred without notice, after which this version may no
longer be accessible. Submitted to the 2023 IEEE/RAS International Conference
on Humanoid Robots (Humanoids). Supplementary video available at
https://youtu.be/y5O2mRGtsL
Poster: How to Raise a Robot - Beyond Access Control Constraints in Assistive Humanoid Robots
Humanoid robots will be able to assist humans in their daily life, in particular due to their versatile action capabilities. However, while these robots need a certain degree of autonomy to learn and explore, they also should respect various constraints, for access control and beyond. We explore incorporating privacy and security constraints (Activity-Centric Access Control and Deep Learning Based Access Control) with robot task planning approaches (classical symbolic planning and end-to-end learning-based planning). We report preliminary results on their respective trade-offs and conclude that a hybrid approach will most likely be the method of choice
How to Raise a Robot - A Case for Neuro-Symbolic AI in Constrained Task Planning for Humanoid Assistive Robots
Humanoid robots will be able to assist humans in their daily life, in particular due to their versatile action capabilities. However, while these robots need a certain degree of autonomy to learn and explore, they also should respect various constraints, for access control and beyond. We explore the novel field of incorporating privacy, security, and access control constraints with robot task planning approaches. We report preliminary results on the classical symbolic approach, deep-learned neural networks, and modern ideas using large language models as knowledge base. From analyzing their trade-offs, we conclude that a hybrid approach is necessary, and thereby present a new use case for the emerging field of neuro-symbolic artificial intelligence
BlueSky: Combining Task Planning and Activity-Centric Access Control for Assistive Humanoid Robots
In the not too distant future, assistive humanoid robots will provide versatile assistance for coping with everyday life. In their interactions with humans, not only safety, but also security and privacy issues need to be considered. In this Blue Sky paper, we therefore argue that it is time to bring task planning and execution as a well-established field of robotics with access and usage control in the field of security and privacy closer together. In particular, the recently proposed activity-based view on access and usage control provides a promising approach to bridge the gap between these two perspectives. We argue that humanoid robots provide for specific challenges due to their task-universality and their use in both, private and public spaces. Furthermore, they are socially connected to various parties and require policy creation at runtime due to learning. We contribute first attempts on the architecture and enforcement layer as well as on joint modeling, and discuss challenges and a research roadmap also for the policy and objectives layer. We conclude that the underlying combination of decentralized systems\u27 and smart environments\u27 research aspects provides for a rich source of challenges that need to be addressed on the road to deployment